AI supports the German government. "If we don't do it now, we'll be behind in five years."

• The IT Department of the Federal Foreign Office, in cooperation with Bundesdruckerei GmbH, has developed the data and AI platform PLAIN.
• The solution is intended to respond to the needs of the administration – it helps build applications and conduct data analytics useful, for example, in negotiations with the European Commission.
• "Instead of relying solely on solutions delivered to our doorsteps, we realized that we had to invest in developing our own staff, otherwise we would never keep up with technology," says Hans Christian Mangelsdorf, chief data scientist at the German Ministry of Foreign Affairs, which developed the platform, in an interview with WNP.
• "In 2022, the federal government decided to establish so-called data labs, i.e., data and AI laboratories, in every ministry. Now, each federal ministry has at least 5-10 data engineers and data scientists. As a result, we now have a network of 100-120 people recruited from the market," adds Mangelsdorf.

The German Federal Foreign Office has created a government platform called PLAIN . What exactly is it and what problem was it intended to address?

"As early as 2022, we were certain there was enormous demand for artificial intelligence solutions—especially chatbots, systems based on retrieval augmented generation (RAG), and others. That's when we made the strategic decision to create our own government cloud that would help us build and share such solutions. So, as the Ministry of Foreign Affairs, we made a greenfield investment: we built our own data center and built a full-fledged cloud, similar to what the largest hyperscalers offer."

We wanted to have such a solution, but run in a sovereign way to ensure that it would work no matter what.

Why do you call it a sovereign cloud if you use solutions from American companies, e.g. HPE?

"We can swap components across layers—hardware, software, applications—and easily replace them if necessary. If a software component is no longer available, we simply change it or use open source from scratch. This is something we strongly promote as a government."

As a side note, it's worth noting that HPE has established assembly lines and service centers in the EU to reduce its international dependence. Other companies are following similar steps.

Why was the PLAIN platform developed by the Ministry of Foreign Affairs? It's a rather unusual ministry when it comes to building digital solutions.

"That's a very good question. In Germany, at the federal level, we essentially have three ministries that have significant IT expertise or manage IT resources. There's the Ministry of Defense, which develops solutions for the military, the Ministry of the Interior, and—for about six months now—the Ministry for Digitalization and Modernization of the State. The latter two are solely responsible for internal, national digitalization."

However, the Federal Foreign Office in Germany has a clear mandate to provide IT services related to our work abroad. Since it's difficult in practice to separate what we do abroad from what we do at home, we saw this as a starting point: let's tackle the problems holistically, as a government.

So then you decided to do it for the entire administration?

"About six years ago, I had a team of data scientists and engineers who were doing very interesting things, like foresight analyses related to crises. We were conducting our own modeling exercises, trying to "predict" the future. We were building dashboards and simulation tools. But the team was doing all of this on their own laptops."

We wanted to demonstrate this and put it into practice in an organization that operates not only within the country but in 192 countries. We needed an infrastructure that would allow us to rapidly prototype and implement it. This was the starting point.

And you thought it could be scaled to the entire government?

"Exactly. We've developed a massive DevOps tool. You can use it to do everything from training large language models, to building dashboards, to deploying any software you deem appropriate."

We've configured this platform to make life easier for those working with data and artificial intelligence. We don't use it for citizen-facing services, such as digital wallets or financial and tax systems, because those require different technology.

"If we don't do it now, in five years we will be completely technologically behind"

Do you train your own language model (LLM)?

We're not yet training our own models from scratch, but we can fine-tune existing ones. We use Gamma 3, Mistral, in the past we used Luminous from Aleph Alpha, and recently we've been using LLAMA 3. We could also use Chinese models on our platform because we ensure it's closed and doesn't communicate externally.

This means you make sure that data does not flow outside the models.

"Exactly, we operate on a closed cloud. This is a necessity in a government environment, where on the one hand you want to enable employees to use such tools, but on the other you need to limit the risk of data and information leakage."

You decided to build your own infrastructure, which was likely very expensive. Why didn't you use other providers? I'm not even talking about hyperscalers, but smaller, European companies.

The answer is simple: from the very beginning, we treated this program as a testing ground and a tool for improving the skills of our own employees. Instead of relying solely on solutions delivered to our doorsteps, we realized that we needed to invest in developing our own staff, otherwise we would never keep up with technology.

We also treat this project as a major upskilling exercise for our teams – to transition from more traditional infrastructure management to a fully containerized cloud.

What were the biggest barriers to implementing the platform in such a traditional institution as the German government?

"True, government is a traditional institution. First, you have to recruit the right people. And then attract them to work in government. It's easy to find DevOps engineers on the market, but getting them to want to work for the federal government and really feel like they're part of the project is a challenge. And if you attract them, you have to retain them. This is a problem for all governments, by the way. But it's a necessary one, because if we don't do it now, in five years we'll be completely technologically behind."

Secondly, silo thinking had to be broken. In the past, it was extremely difficult to convince other government institutions that it was worthwhile to join forces rather than do everything separately.

We know it from Poland.

"The topic of silos keeps coming up; that's the crux of the problem. However, for us, a strong argument for breaking this mechanism came in 2023, when suddenly everyone in the administration wanted to buy GPUs. They rushed to the market, trying to get two, three, four units of the NVIDIA H100. And we said, 'Don't worry, guys, the problem has already been solved.' We have 50 such units in our data center."

Because you started building them a year earlier.

"And thanks to that, we were a step ahead of the others. We could tell our colleagues: 'We've got this covered.'" Later, however, we had to convince them that it made sense to use a shared infrastructure where they could actually collaborate.

Today , we have ministries that actually work on our platform, sharing code. They create their own branches in GitLab, then merge them, and publish applications available to the entire government. But it's worth honestly saying that we already had a certain advantage.

Meaning?

In 2022, the federal government decided to establish so-called data labs, or data and AI laboratories, in every ministry. Now, each federal ministry has at least 5-10 data engineers and data scientists. As a result, we now have a network of 100-120 people recruited from the market, who have a completely different mentality from that of bureaucrats – one based on collaboration.

For these people, it's natural not to do everything yourself. They say: let's use open source, let's work together. Let the Ministry of Foreign Affairs handle the AI ​​chatbot, and we'll handle the simulation tool. Then let's put it all into a shared GitLab and work independently and securely.

In Poland, every ministry has an IT department, but in a very traditional sense. In addition, there are several government software houses.

"And that's where the difference lies: IT, in the traditional sense, focuses on maintaining systems, which is and will continue to be important because it provides the basic infrastructure. However, in the new system, the team is supposed to inspire superiors. A minister or deputy minister might come in and say, 'I need data analysis on our global trade relations or on risks in a given country.' And that triggers the team's actions."

It doesn't happen often, but there are rare moments in the history of government when truly great ideas are implemented. In my opinion, this was one of those moments. Today, we see that if you have the right people, the right vision, and the right tools at the right time, distributed teams can do truly fascinating things. And at a very fast pace.

What real problem can they solve?

- For example, responses to parliamentary questions. In Germany, the government must answer questions from members of parliament.

Here too, it's a scale of 40,000 during the term of office.

"We have closer to 40,000 a year. It's often a data issue—it needs to be collected and processed, and potentially, AI could be used to generate answers. We're not doing that right now, but it's a great example of how you can quickly create a Pareto-type solution that works well enough."

So you use the platform for decision-making support?

Some ministries use platforms for managing data within their organizations—classic dashboards. Others, including us, use them to implement secure LLM-based chatbots, summarization tools, and translations. Others use them for simulations, such as in international negotiations or budget talks with the European Commission. The common denominator is always working with large data sets, for which previously appropriate tools were lacking.

Because the typical government employee has a laptop with an office suite—and that's not the place for programming and deploying applications. Now they have access to a platform they can use from virtually any system.

"Instead of pumping billions, it's better to use public money to support companies"

Has there been any controversy surrounding the system's implementation, either in society or in the media? For example, questions about GDPR or data use?

There were no vocal criticisms, but rather questions. They mainly concerned information security, which was one of the main reasons for initiating the project. There were also questions related to GDPR, especially from employee representatives.

This was a great opportunity to explain to colleagues how cloud infrastructure works. While it's GDPR-neutral, we build specific applications on the platform that may use additional data and, in the future, could become high-risk systems. Therefore, it was important for employee representatives to understand that they shouldn't be concerned about the platform itself, but rather about specific use cases.

And compliance with the AI ​​Act – is that a problem for you?

No, because the platform itself is agnostic. It can use any model. And when it comes to applications, we try to keep them as small as possible.

So there are no problems with high-risk systems?

"They might emerge, for example, when we start using AI to analyze visa applications. This wouldn't automatically be considered high-risk AI, but it would require documentation and security measures."

Do the police also use the platform?

"No, it doesn't yet. It focuses on private clouds, completely isolated from the internet. Our product is a community cloud—with controlled, secured external access, because developers have to download container images or publish code. But security services prefer completely isolated systems. They may use parts of our platform someday, but it will never be the only solution."

Can PLAIN be replicated in other countries or at the European Commission level?

- Yes. We're already talking about this with our partners, and we'll be having meetings on this matter at the end of this year.

Isn't it possible that Europe has overslept in developing similar solutions? In two or three years, won't it be too late?

No. In my personal opinion, we missed the opportunity at the level of building hyperscalers. However, the issue of sovereign software solutions is still open. Almost everything needed to support the cloud exists in open source.

So Europe can build an alternative to the US and China?

Yes, although it's important to remember that the problem isn't the technology, but the business model. The hyperscaler market is already occupied. Entering it is a huge expense. So instead of pumping in billions, it's better to use public money to support companies in creating sustainable, European business models.

How can countries like Poland build digital sovereignty with smaller budgets than Germany?

- Focus on your strengths. There's no point in 15 countries in Europe doing the same thing, because each would be too small. But if we join forces, it's possible.

Is Germany striving for full digital sovereignty?

"No, that's impossible. There are indeed some moves, such as moving away from specific software requiring licenses in schools. But it's really a matter of user acceptance. Some German governments switched to Linux and open source 20 years ago, and then returned to MS Office. Technically, anything is possible, but the key is whether users accept it."

However, we do have some interesting initiatives, such as OpenDesk, created by the ZenDis Center for Digital Sovereignty – entirely open source, with European technical support. This could be a viable alternative in a year or eighteen months.

What are the next milestones for the PLAIN program?

- First, make it available to all federal agencies that want to use it.

Secondly, building joint projects involving several ministries at the same time – for example, around AI – and making them available to all employees.

Third, solving the problems associated with software development in "containers." Today, developers download images from GitHub, ignoring security indicators (CVEs). And if you want to move from experimentation to production, you need to control it. Therefore, at the upcoming Smart Country Convention in Berlin, we will announce a partnership with the Center for Digital Sovereignty, which will maintain and share 40-50 of the most important base images in a secure, continuously updated version—available to the entire government.